{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e37ea7a6",
   "metadata": {},
   "source": [
    "# Intel® Neural Compressor Sample for Tensorflow"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "799b2d67",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "This is a demo to show an End-To-End pipeline to speed up AI model by Intel® Neural Compressor.\n",
    "\n",
    "1. Train a CNN AlexNet model by Keras and Intel Optimization for Tensorflow based on dataset MNIST.\n",
    "\n",
    "2. Quantize the frozen PB model file by Intel® Neural Compressor to INT8 model.\n",
    "\n",
    "3. Test and compare the performance of FP32 and INT8 model by same script.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e538cd95-f291-41aa-9b48-00956855aec1",
   "metadata": {},
   "source": [
    "## Code\n",
    "Please refer to [README.md](README.md)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71858ff2-c7b6-425e-a7c4-eff227cc481e",
   "metadata": {},
   "source": [
    "## Prepare Running Environment\n",
    "\n",
    "Please refer to [README.md](README.md)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "735982ec-2398-479b-a927-01d7e9f30ea1",
   "metadata": {},
   "source": [
    "### Remove all old output files (Optional)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c7ca46e-0fc8-4818-ac57-d354414ee6d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "!rm -rf run_in_intel_devcloud.sh.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82f09276",
   "metadata": {},
   "source": [
    "## Run in Intel® DevCloud\n",
    "\n",
    "Job submit to compute node with the property 'clx' or 'icx' or 'spr' which support Intel® Deep Learning Boost (avx512_vnni)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f51bc091",
   "metadata": {},
   "outputs": [],
   "source": [
    "!qsub run_in_intel_devcloud.sh -d `pwd` -l nodes=1:icx:ppn=2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f0d7cab-1b60-4689-b153-506e5818b811",
   "metadata": {},
   "source": [
    "Check job status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7199754d-e7e4-4e52-868d-0a1ca79cb064",
   "metadata": {},
   "outputs": [],
   "source": [
    "!qstat"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bc5b605-47d1-485f-bfb1-cd7ab9f3f83c",
   "metadata": {},
   "source": [
    "### Check Result\n",
    "\n",
    "#### Check Result in Log File\n",
    "Check the latest created log file with prefix: **run_in_intel_devcloud.sh.o**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6b36c9c-f612-4517-914c-d5ca6ee92d27",
   "metadata": {},
   "outputs": [],
   "source": [
    "!tail -23 `ls -lAtr run_in_intel_devcloud.sh.o* |  tail -1 | awk '{print $9}'`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9822cad-22e8-415d-b33b-fd12065c5163",
   "metadata": {},
   "source": [
    "Check any existed log file, for example **run_in_intel_devcloud.sh.o1842343**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d49e69a-ccfe-42f0-982d-38395ed4a808",
   "metadata": {},
   "outputs": [],
   "source": [
    "!tail -23 run_in_intel_devcloud.sh.o1842343"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a80934c4-8ddd-48c3-acc5-63dc0bb1372a",
   "metadata": {},
   "source": [
    "#### Check Result in PNG file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44c31db2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image, display\n",
    "\n",
    "listOfImageNames = ['fp32_int8_aboslute.png',\n",
    "                    'fp32_int8_times.png']\n",
    "\n",
    "for imageName in listOfImageNames:\n",
    "    display(Image(filename=imageName))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b4cded5-3723-42e5-aec1-8ec514ccd49e",
   "metadata": {},
   "source": [
    "## Run in Customer Server or Cloud\n",
    "\n",
    "Note, it's recommended to use 2nd Generation Intel® Xeon® Scalable Processors or newer to get better performance improvement.\n",
    "\n",
    "### Run in Jupyter Notebook\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "741279c7-f788-47f1-ab9a-8f0628a79d16",
   "metadata": {},
   "outputs": [],
   "source": [
    "!./run_sample.sh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d53543c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3cb8011-31c4-4a7c-be00-775d2ec940f4",
   "metadata": {},
   "source": [
    "### Check Result\n",
    "\n",
    "#### Check Result in Screen Output\n",
    "\n",
    "```\n",
    "...\n",
    "\n",
    "Compare the Performance of FP32 and INT8 Models\n",
    "Model            FP32                     INT8                    \n",
    "throughput(fps)  572.4982883964987        3218.52236638019        \n",
    "latency(ms)      2.8339174329018104       1.9863116497896156      \n",
    "accuracy(%)      0.9799                   0.9796                  \n",
    "\n",
    "Save to fp32_int8_aboslute.png\n",
    "\n",
    "Model            FP32                     INT8                    \n",
    "throughput_times 1                        5.621889936815179       \n",
    "latency_times    1                        0.7009066766478504      \n",
    "accuracy_diff(%) 0                        -0.029999999999986926   \n",
    "\n",
    "Save to fp32_int8_times.png\n",
    "Please check the PNG files to see the performance!\n",
    "This demo is finished successfully!\n",
    "Thank you!\n",
    "...\n",
    "\n",
    "```\n",
    "#### Check Result in PNG file\n",
    "\n",
    "The demo creates figure files: fp32_int8_aboslute.png, fp32_int8_times.png to show performance bar. They could be used in report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82c4f0b7-2451-41db-bd84-0fc26e74aab2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image, display\n",
    "\n",
    "listOfImageNames = ['fp32_int8_aboslute.png',\n",
    "                    'fp32_int8_times.png']\n",
    "\n",
    "for imageName in listOfImageNames:\n",
    "    display(Image(filename=imageName))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f882b021-190d-438e-9cc8-f76b501c6be5",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  },
  "nbTranslate": {
   "displayLangs": [
    "*"
   ],
   "hotkey": "alt-t",
   "langInMainMenu": true,
   "sourceLang": "en",
   "targetLang": "fr",
   "useGoogleTranslate": true
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
